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ABSTRACT 


Much  of  the  redundancy  in  a  speech  or  television  signal  is  elimi¬ 
nated  when  that  signal  is  encoded  into  digital  form  by  a  differential 
PCM  encoder.  Further  coding  of  the  differential  PCM  output  using 
entropy  coding  techniques  (Huffman  or  Shannon-Fano  coding)  can  result 
in  a  further  increase  in  the  signal  to  quantizing  noise  ratio  of  5.6  dB 
without  increasing  the  transmission  rate.  This  conclusion  is  reached 
by  comparing  the  performance  of  a  DPCM  system  without  entropy  coding 
with  one  using  entropy  coding.  The  DPCM  without  entropy  coding  uses 
a  minimum  mean  square  error  quantizer  (Max  quantizer)  while  the  system 
with  entropy  coding  uses  an  equilevel  quantizer.  The  DPCM  quantizer 
inputs  are  assumed  to  have  a  Laplacian  probability  density  function. 
This  is  consistent  with  previous  experiments  with  speech  and  television 
signals.  Gaussiar  quantizer  inputs  result  in  only  a  2.8  dB  improvement 
for  entropy  coding. 
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Much  of  the  redundancy  in  a  speech  or  television  signal  is  eliminated  when 
that  signal  is  encoded  into  digital  form  by  a  differential  PCM  encoder.  Further 
coding  of  the  differential  PCM  output  using  entropy  coding  techniques  (Huffman  or 
Shannon-Fano  coding)  can  result  in  a  further  increase  in  the  signal  to  quantizing 
noise  ratio  of  5.6  dB  without  increasing  the  transmission  rate..  This  conclusion 
is  reached  by  comparing  the  performance  of  a  DPCM  system  without  entropy  coding 
with  one  using  entropy  coding.:  The  DPCM  without  entropy  coding  uses  a  minimum 
mean  square  error  quantizer  (Max  quantizer)  while  the  system  with  entropy  coding 
uses  an  equilevei  density  function.  This  is  consistent  with  previous  experiments 
with  speech  and  television  signals.  Gaussian  quantizer  inputs  result  in  only  a 
2.8  dB  improvement  for  entropy  coding. 
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ISn&EXXTlOS 


Differencial  PCM  (DFCJt)  is  an  efficient  way  to  encode  highly 
correlated  analog  signals  into  binary  form  suitable  for  digital  trans¬ 
mission,  storage  or  for  input  to  a  digital  ccsputei .  Based  primarily 
on  a  patent  by  Cutler  in  1952  [l],  DPCM  systems  have  been  studied 
extensively  for  encoding  television  [2,3,-«J  and  speech  [5,6]. 

The  qua  I! wZci s  used  in  i-’St  previ  yes  studies  of  DPCM  have  been 
quantizers  tapered  in  various  ways  to  march  the  amplitude  density  of 
the  signal  to  be  quantized.  Such  quantizers  were  originally  studied 
by  Panter  and  Dite  [7]  ar.d  an  algorithm  for  their  design  was  presented 
by  Max  [8].  Nitadori  [ 5 ]  and  O’Neal  [4]  found  that  for  speech  and 
television,  respectively,  well  designed  DPCM  systems  had  quantizer  in¬ 
put  signals  whose  probability  density  functions  were  approximately 
Laplacian  (two-sided  exponential).  Paez  and  Glisson  [9]  apply  Max's 
technique  to  the  design  of  quantizers  for  Laplacian  signals.  Quantizers 
designed  usinfe  Max's  technique  minimize  the  quantizing  noise  for  any 
given  number  of  quantizing  levels  and  will  be  called  Optimum  Fixed  N 
Quantizers  (N  =  number  of  quantizing  levels).  Wood  [10]  and  Gish  and 
Pierce  [11]  have  recently  shown  that  these  quantizers  are  not  optimum 
whenever  entropy  coding  is  used  to  further  encode  the  quantizer  output 
but  that  uniform  quantizers  (with  equally  spaced  levels)  are  very  nearly 
optimum  when  used  in  conjunction  with  entropy  coding.  In  this  paper  we 
investigate  the  utility  of  entropy  coding  for  DPCM  by  comparing  the 
performance  of  the  two  coders  shown  in  Figure  1.  The  performance  metric 
to  be  used  is  the  signal  to  quantizing  noise  ratio  at  any  given  baud 
rate  in  the  channel. 
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This  study  leads  to  the  conclusion  that  -litre  jy  coding  can  increase 
the  signal  to  quantizing  noise  ratios  of  D?C •»  .or  speech  and  television 
by  as  such  as  5.6  dB  when  the  baud  rare  is  '  fxge-  For  lover  baud  rates 
the  gain  is  somewhat  less  —  e.g.,  at  a  ctu  • jei  baud  rate  of  six  tines 
the  bandwidth  a  gain  of  ~ .4  dB  is  possibt'  *ith  entropy  coding.  These 
results  agree  qualitatively  with  those  of  ‘  tow  1 1° j  who  studied  the 
statistics  of  a  4  bit  DPCM  coder  for  Pic*. ur ephone.  It  is  important  to 
realize  that  entropy  ceding,  vhich  alio  ,  transmission  at  a  rate  equal 
to  the  entropy  of  the  quantizer  output,  equires  buffering  of  tne  bit 
stream  and  the  use  of  variable  length  code  words.  Such  a  coding  tech¬ 
nique  is  difficult  to  implement  and  th j  full  5.6  dB  improvement  from 
entropy  coding  will  be  difficult  to  achieve  with  practical  hardware. 

The  Oper-ition  of  DPCM 

The  operation  of  the  DPCM  systems  of  Figure  1  can  be  summarized 

1  2 
as  follows:  The  input  signal  S(t)  whose  mean  square  value  is  a  ,  is 

sampled  at  twice  its  bandwidth  W  to  produce  a  sequence  of  sample  values 
SQ,  S^,  ...  =  {S^K  At  the  same  time,  the  predictor  makes  an  estimate 
of  each  sample  value  based  on  those  which  have,  preceeded  it.  These 
estimates  are  the  sequence  S^,  Sj,  .,.  =  i S ^  } .  Each  estimate  is  sub¬ 
traced  from  the  actual  sample  value  producing  a  difference  or  error 
sequence  e^,  ...  =  <e^,  where  =  S_^  -  S  .  The  prediction  process 

in  DPCM  causes  the  members  of  the  sequence  >  1  to  be  independent.  The 

quantizer  represent  each  value  of  e^  by  the  nearest  quantizing  level. 
The  quantizing  levels  are  transmitted  and  used  by  the  receiver  to 

^For  a  more  detailed  discussion  of  DPCM  see  [4]. 


reconstruct  the  original  analog  signal.  The  difference  between  each  e^ 

and  the  quantizing  level  used  to  represent  it  is  called  quantizing  error 

The  quantizing  levels  actually  transmitted  are  the  sequence  {e^  -  q^> 

and  this  is  the  sequence  used  by  the  decoder  to  reconstruct  the  analcg 

signal.  The  quantizing  error  shows  up  in  the  reconstructed  signal  as 

quantizing  noise  q(t).  The  decoder  output  is  thus  S(t)  +  q(t_).  When 

2 

the  number  of  quantizing  levels  N  is  large  (N  >_  c  is  large  enough)  , 
the  mean  square  value  of  q(t),  is  the  same  as  the  mean  square  value  of 
{q^} .  The  signal  to  quantizing  noise  of  the  DPCM  system  can  be  expressed 


2  2  a  2  , 

2_  =  5__®=5__L  n 

2  2  2  “  2  2  u 

o  a  a  a  e 
q  e  q  e 

2 

2  ° 

where  e  =  — ^  is  the  normalized  quantizing  noise  power  and  may  be 
a 

e 

thought  of  as  the  quantizing  noise  power  produced  when  the  quantizer 

2 

input  (e^)  has  unit  variance.  The  quantity  e  is  determined  by  the 

properties  of  the  quantizer  and  the  probability  density  function  of 

a2 

(e^K  The  quantity  —  represents  the  amount  by  which  the  power  of 


the  signal  can  be  reduced  by  prediction.  It  is  a  measure  of  the  redun¬ 
dancy  in  the  signal  S(t)  and  can  be  determined  from  its  statistics. 

a2  2 

This  quantity  —  will  be  the  same  for  both  systems  in  Figure  1  hut  e 

e 

will  depend  on  whether  or  not  entropy  coding  is  used.  When  the  baud 


4 


rate  is  large  enough  — ^  and  are  relatively  independent  and  can  be 

"e 

calculated  separately  —  this  is  the  case  we  are  dealing  with  in  this 
2 

paper. 


Relationship  Between  Quantizing  Noise 
and  Baud  Rate 

The  baud  rate  B  used  to  transmit  the  quantizing  levels  through 
a  binary  channel  for  the  DPCM  system  without  entropy  coding  —  in 
Figure  1(a)  —  is  simply  log-  N  times  the  sampling  rate  2W,  i.e,, 

c  2i 

Ba  -  2W  log2  Na,  (2) 


where  N  is  the  number  of  quantizing  levels.  When  the  input  to  the 
quantizer  {e^}  is  a  Laplacian  signal  of  unit  variance  (i.e.,  one  whose 
probability  density  function  is  p(x)  =  exp  (-V2  |x]  )),  the  mean 

v  2 

square  quantizing  error  approaches  (see  Equation  (15)  of  Reference  [4]), 
as  N  gets  large,2 


2 
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(3) 


a 

Using  (3)  in  (2)  gives  the  baud  rate  in  terms  of  the  quantizing  noise 


B 

a 


;  W  log 


2  2c2 
a 


(4) 


2  o  2 

For  low  baud  iates  where  —z  and  £  cannot  be  calculated  inda- 

o 

e 

pendently >Gish  [13]  has  developed  a  formula  for  DPCM  which  applies  to 
first  order  Markov  signals. 

3 

In  this  paper  the  expression  A  B  means  that  A  is  approximately 
equal  to  B  and  furthermore  lim  A-B  where  N  is  the  number  of  quantizing 
levels.  NVJO 


5 

For  the  DPCM  systes  with  entropy  coding  —  in  Figure  1(b)  —  the 
baud  rate  in  the  channel  is  equal  to  the  sampling  rate  Multiplied  by 
the  entropy  of  the  quantizer  output  H, 


B,  =  2  WE.  (5) 

b 

Wood  [10]  has  shown  chat  when  N^,  the  numbei  of  quantizing  levels,  gets 
large  H  approaches 

H  '  H.  -  log^A  (6) 


where  A  is  the  step  size  of  the  uniform  quantizer  and  K.  is  the  entropy 

2 

of  the  quantizer  input.  The  mean  square  quantizing  error  is  related 

2  12 

to  the  step  size  by  --  —  &  anci  *or  Laplacian  input  signals  is 


OF, 

Hi  -  /  ”  p(x)  log2  p(x)  dx  =  log2  /2e. 

—  y< 


(7) 


Making  these  substitutions  in  (6)  and  (5)  gives  the  baud  rate  in  terms 
of  the  mean  square  quantizing  error 
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2 

"  W  log2 
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(8) 


To  compare  the  signal  to  quantizing  noise  ratios  of  the  two  systems 
when  their  baud  rates  are  equal  we  set  (8)  equal  to  (4)  giving 
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0.272  e2 
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(9) 


This  is  the  relationship  between  the  quantizing  noise  powers  when  the 
two  systems  are  operating  at  the  same  baud  rate.  This  equation  says 
that  when  the  baud  rate  is  large  a  quantizing  system  using  entropy 
coding  can  achieve  a  signal  to  quantizing  noise  ratio  of  5.64  dB 
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2 

(-10  log  greater  than  a  system  without  entropy  coding.  This  applies 
to  DPCM  through  Equation  (lj . 

What  happens  at  lover  baud  rates  is  shown  in  Figure  2.  The  curve 
for  the  case  of  entropy  coding  was  plotted  from  (8).  The  curve  for  the 
case  without  entropy  coding  was  plotted  frem  results  of  Paez  and  Glisson 
[9]  which  show  the  relationship  between  t  and  N  for  small  N  .  Our 

Si  <1  Sk 

equation  (3)  is  inaccurate  for  values  of  N  less  than  about  64.  In  Figure 

1  3  n1 

2  the  quantity  — -  in  (1)  was  assumed  to  be  14  db  lie.,  10  log  — j  -  14/. 

0  J 

e  e 

This  means  that  the  mean  square  value  cf  the  signal  can  be  reduced  by  14 

dB  by  prediction  —  this  assumption  ic  reasonably  consistent  with  pre¬ 
vious  experiments  with  speech  and  television  [4,6] . 

The  above  calculations  are  based  on  Laplacian  signals  because  pre¬ 
vious  studies  showed  that  for  speech  and  television  signals  DPCM  quantizer 
inputs  are  approximately  Laplacian.  These  calculations  are  easily  extended 
to  Gaussian  signals  as  follows.  In  the  case  of  Gaussian  signals  (3)  and 
(4)  become,  respectively,  for  large  baud  rates, 
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For  entropy  coding  (7)  and  (8)  become,  respectively 


H. 

l 


1  °&2 


(ID 


(12) 


B 


b 


W 


log. 


ne 


6  c 


2 

b 


(13) 


7 


Setting  the  baud  rates  equal,  (11)  =  (13),  gives 

e?  ~  — — —  t1  -■  0.523  t2  . 
b  3j,7  a 

g 

Thus  for  Gaussian  signals  entropy  coding  gives  a  2.81  dB  (-10  log  — -) 

3/3 

reduction  in  quantizing  noise  over  systems  without  it.  Comparing  (13) 

with  Shannon's  rate  distortion  function,  R  =  W  log2  for  a  unit 

e 

variance  Gaussian  signal  shows  that  at  =  R,  ^  .  This  means 

that  for  large  Daud  rates  quantizing  systems  with  entropy  coding  give 
quantizing  noise  only  1.53  dB  (10  log  above  what  is  possible  for 
any  encoding  system. 

The  results  of  this  paper  apply  to  DPCM  systems  which  do  not  change 
or  adapt  themselves  to  the  statistics  of  the  input  signal.  In  adaptive 
systems  the  quantizer  levels  may  change  in  accordance  with  certain  proper¬ 
ties  of  the  quantizer  input  signal.  The  results  of  this  paper  may  not 
apply  to  adaptive  systems. 

The  5.6  dB  gain  predicted  for  entropy  coding  assumes  that  the 
quantizer  inputs  are  Laplacian.  There  is  some  evidence  [6]  that  suggests 
hat  for  speech,  the  DPCM  quantizer  input  is  more  peaked  than  the 
Laplacian  function  and  may  be  more  accurately  represented  by  a  Gamma 
distribution.  In  this  case  the  gain  for  entropy  coding  could  be  greater 


than  the  5.6  dB  calculated  above. 
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Figure  2.  Signal  to  quantizing  noise  ratios  S/N  for  the  two  DPCH 
systems  in  Figure  1.  Both  systems  assume  a  prediction 

gain  of  14  dB,  i_.£. ,  in  (1)  10  log  o^/o^  •  14. 
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